1. Field of Invention
Embodiments of the present invention relate to eyetracking systems and methods that compensate for physiological variations in the location of the pupil within the eye.
2. Background Information
Eyetracking has application in many areas of technology, including software applications used to assist those with disabilities, military uses, and space programs. Generally, eyetracking has application in any system where it is desired to know where a user is looking.
In known video eyetracking methods, the angular orientation, or gaze direction, of the eye is measured via analysis of an image of the eye. Typically, the image is made with a camera. Referring to FIG. 1, a video camera 4 observes an observer""s eye 1, which is illuminated by a light source 3. The light source 3 (e.g., an infrared light-emitting diode) illuminates the eye 1 as the viewer looks at a screen 2. FIG. 2 illustrates an example of an image of the eye 1 as seen by a camera. The resulting camera image of the eye 1 includes the iris, the pupil 5 (with a center 10) and a corneal reflection 6, i.e., a reflection of the light source off the reflective corneal surface of the eye.
Given that the cornea is approximately spherical in shape, the corneal reflection moves continuously across the eye as the eye changes its orientation. Using the Pupil Center Corneal Reflection (PCCR) method, as disclosed in the references U.S. Pat. No. 3,462,604 to K. Mason, Control Apparatus Sensitive to Eye Movement; John Merchant and Richard Morrissette, A Remote Oculometer Permitting Head Movement, Report No. AMRL-TR-73-69 (Honeywell Radiation Center 1973); and Laurence R. Young and David Sheena, Methods and Designs: Survey of Eye Movement Recording Methods, Behavior Research Methods and Instrumentation, Vol. 7(5), 397-429 (1975), the eye""s orientation is determined from the relative locations of the pupil center 10 and the corneal reflection 6 within the camera image (see FIG. 2). It is known that the vertical and horizontal orientation of the eye is directly related to the vector distance between the corneal reflection 6 and the pupil center 10. For example, as the eye rotates upward, the pupil 5 (and thus the pupil center 10) moves upward with respect to the corneal reflection 6, and as the eyeball rotates toward the camera""s right, the pupil center 10 moves right with respect to the corneal reflection image.
The corneal reflection is often referred to as the glint spot, and the vector between the glint spot and the pupil center is often referred to as the glint-pupil vector:
vxxpupilxe2x88x92xglintxe2x80x83xe2x80x83(1a)
vy=ypupilxe2x88x92yglintxe2x80x83xe2x80x83(1b)
where:
vx, vy are the horizontal and vertical components of the glint-pupil vector (input variable set)
xpupil, ypupil are the x,y location of the pupil-center image within the camera frame
xglint, yglint are the x,y location of the corneal-reflection image within the camera frame.
Known eyetracker systems and methods commonly use mathematical equations to predict the eye orientation from the measured glint-pupil vector. A simple example of a known equation to predict the eye orientation is:
Thetahorz=Ax0+Ax1vxxe2x80x83xe2x80x83(2a)
xe2x80x83Thetavert=Ay0+Ay1vyxe2x80x83xe2x80x83(2b)
where:
Thetahorz, Thetavert are the horizontal and vertical angles of the eye orientation, and Ax0, Ax1, Ay0, Ay1 are the coefficients used in predicting the eye""s orientation (calibration parameter set).
The variables from which the eye orientation is computed are commonly referred to as the xe2x80x9cinput variable setxe2x80x9d. In the example of Equations 2a, 2b above, the input variable set consists of the variables (vx and vy). The coefficients, used to compute the eye orientation from the input variable set, are often referred to as the xe2x80x9ccalibration parameter setxe2x80x9d. In the example of Equations 2a, 2b above, the calibration parameter set consists of the coefficients (Ax0, Ax1, Ay0, and Ay1).
Typically, a xe2x80x9ccalibration procedurexe2x80x9d is used to determine optimum values for calibration parameters. One known calibration procedure consists of the user fixating on a sequence of known locations. A set of xe2x80x9crawxe2x80x9d data is collected at each calibration point, where the raw data consists of the known gaze directions and a set of measured input data for each of the calibration points. After the raw data for all the calibration points has been collected an optimization procedure, such as a regression analysis, is executed to find optimal values for the calibration parameters, which yield a minimum aggregate error in predicting the known gaze directions for all the calibration points.
Problem Predicting Gaze Direction Accurately
When using the PCCR method to measure the orientation of an eye, it is assumed that the pupil center represents a fixed point within the eyeball structure. A problem exists, however, in that as the iris dilates and constricts, the pupil typically does not open and close completely concentrically about a fixed point on the eye. An exaggerated example of an eye with pupil-center shift is illustrated in FIG. 3. FIG. 3 illustrates an example of how two pupils are not concentric; the two pupil centers (9, 10) are at different locations within the eye. Pupil center 9, the center point of pupil circumference 8 is offset from pupil center 10, the center point of pupil circumference 7.
This pupil center shift can be explained as follows: The iris contracts the dilates using two opposing muscles. The sphincter pupillae, oriented circumferentially around the inner edge of the iris, effects contraction by pulling the pupil closed. The dilator pupillae, oriented radially, effects dilation by pulling the pupil open. If the sphincter and/or dilator pupillae do not actuate symmetrically around the pupil, or if there is an imbalance in the counteracting push/pull mechanics of the iris""s sphincter and dilator muscles, the pupil center shifts within the eyeball.
A problem exists because of the extent that the pupil-center location varies within an eyeball, the glint pupil vector (as measured within the camera image) will not represent a vector from the corneal reflection to a fixed point on the eyeball, and errors result in the calculation of the eyeball orientation, unless these pupil-center shifts are accommodated.
Typically, human pupils open and close concentrically enough to maintain PCCR eyetracking accuracies of approximately 0.7 degrees (using root mean square error measurement). However, a need exists for a more accurate method for cases where more accurate eyetracking is desired, or for people whose pupil center(s) shift more than normal within the eyeball(s). Accordingly, it is desirable to have methods to measure the image location of a more nearly fixed reference point within the eyeball.
Embodiments of the invention include a method of using a computer system to determine an eye""s pupil-offset from a fixed location on the eyeball as a function of its diameter using an eyetracker system and the pupil-center-corneal reflection method to measure glint-pupil vectors of an observer""s eye. Preferably, the observer fixates an eye on one or more given points on a screen and the system is used to measure at least one additional glint-pupil vector of the eye. The system quantifies measured variations in the glint-pupil vector as pupil offsets with respect to a fixed location on the eyeball and codifies them as a function of measured pupil diameters.
Additional embodiments of the invention include a method to determine the orientation of an eye by accommodating for the eye""s pupil center shift, comprising the steps of determining the location of the pupil; measuring at least one observable feature of the eye related to the relative location of the pupil within the eyeball; estimating a pupil-location-offset parameter with respect to a fixed point on the eyeball using at least one said measured observable eye feature; and calculating the location of a fixed point within the eyeball as a function of the measured pupil location and the estimate of the pupil-location offset.
Other embodiments of the invention include a method of using a computer system to quantify an eye""s pupil-offset behavior, comprising the steps of using an eyetracker system and the pupil-center-corneal reflection method to measure a first glint-pupil vector of the eye; fixating the eye on one or more given points on a screen; measuring at least one additional glint-pupil vector of the eye; and quantifying measured pupil offsets as functions of measured pupil diameters.
Still other embodiments of the invention include a system for compensating for pupil non-concentricity in an eyetracker system, comprising a video eyetracker for tracking the position of an observer""s eye; a processor for using the pupil-center-corneal reflection method to measure at least one glint-pupil vector; a means for measuring glint-pupil vectors as the pupil diameter varies; and a processor for interpreting the measured variations in the glint-pupil vector as pupil offsets with respect to a fixed location on the observer""s eyeball and codified as a function of the measured pupil diameters.
Embodiments of the invention also include an apparatus for estimating the gaze direction of an eye comprising a computer and a camera for observing the eye. A measuring means measures pupil diameter and features of the eye related to the relative location of the pupil within the eyeball and a processor estimates the pupil-location-offset with respect to a fixed point on the eyeball using at least one of the measured observable eye features. A processor estimates the location of the fixed point within the eyeball as a function of the measured pupil location and the estimate of the pupil-location offset.